Skip to content

Wait for uploader tasks before actor shutdown - #6868

Draft
fmassot wants to merge 1 commit into
mainfrom
fmassot/uploader-shutdown-drain
Draft

fmassot wants to merge 1 commit into
mainfrom
fmassot/uploader-shutdown-drain

Conversation

@fmassot

@fmassot fmassot commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Ensure joining an uploader also waits for its outstanding storage/metastore writes.

  • Track uploads before dispatch and drain them on shutdown.
  • On handler panic, drain uploads before resuming the original panic.
  • Add held-upload regressions for quit, kill, and panic.

No Parquet-specific changes.

Tests

234 indexing tests passed; clippy, formatting, and license checks passed.

Track uploads before dispatch and drain them on shutdown. If a handler panics, drain outstanding uploads before resuming the original panic. Cover quit, kill, and panic with held-upload regression tests.
@guilload

guilload commented Oct 5, 2026

Copy link
Copy Markdown
Member

Quick feedback on this one:

  • let's honor the decommission timeout (indexers and compactors)
  • let's not handle panics
  • let's add visibility (number of splits, split URIs) on the uploads still in flight

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants